UniNE at CLEF 2006: Experiments with Monolingual, Bilingual, Domain-Specific and Robust Retrieval

For our participation in this CLEF evaluation campaign, the first objective was to propose and evaluate various indexing and search strategies for the Hungarian language in order to produce better retrieval effectiveness than language-independent approach (n-gram). Using both a new stemmer including some derivational suffixes removals, and a more aggressive automatic decompounding scheme, we were able to produce better retrieval effectiveness than corresponding 4-gram indexing scheme. Our second objective was to obtain a better picture of the relative merit of various search engines with the French, Brazilian/Portuguese and Bulgarian languages. To do so we evaluated these test-collections using the Okapi, Divergence from Randomness (DFR) and language model (LM) models together with nine vector-processing approaches. After pseudorelevance feedback, either the DFR or the LM approach tends to produce the best IR performance. For the Bulgarian language, we also found that word-based indexing proposes usually better retrieval effectiveness than corresponding 4-gram indexing. In the bilingual track, we evaluated the effectiveness of various machine translation systems to automatically translate a query submitted in English into the French and Portuguese languages. After blind query expansion, the MAP achieved by the best single MT system is around 95% of the corresponding monolingual search when French is the target language, or 83% with the Portuguese. Using the GIRT corpora (available in German and English), we investigated variations in retrieval effectiveness when facing with domain-specific collection composed of relatively short bibliographic notices. Finally, in the robust retrieval task we investigated different techniques in order to improve the retrieval performance of difficult topics. In this track, we found that both the mean average precision and the geometric mean are strongly correlated. Moreover, massive query expansion based on a search engine did not provide better retrieval effectiveness than Rocchio’s approach.